Prior Art Search and Its Evaluation

Prior Art Search is an information seeking task where searchers, for instance patent examiners, search for published literature to determine whether the claimed invention in a patent application is novel. In Prior Art Search, search tasks are often timesensitive and consist of rich information needs with multiple aspects/subtopics. In this thesis, we explore information retrieval techniques and evaluation metrics for prior art search. The work consists of three parts. Given a patent application document, we rst manage to retrieve relevant patent documents by performing retrieval at both the document level and the passage level. At the document level, we focus on automatically formulating e ective search queries. The queries are formed by extracting terms from claims, titles and hyphenating phrases in a patent application and re ned based on Inverse Document Frequency (IDF) and Part-of-Speech (POS) tagging. At the passage level, we propose a TF-IDF-based retrieval algorithm to calculate the relevance score for each passage and select the most relevant passages. Second, we propose a novel evaluation metric for prior art search. The new evaluation metric, termed the Cube Test (CT), is based on the proposed conceptual user utility model the Water Filling Model, which describes the process of prior art search. We compare our metric with existing prior art search evaluation metrics, as well as existing Web search evaluation metrics in correlation and discriminative power. Experiment results show that our metric e ectively captures the characteristics of prior art search.